Discovering Human-Object Interaction Concepts via Self-Compositional Learning

A comprehensive understanding of human-object interaction (HOI) requires detecting not only a small portion predefined HOI concepts (or categories) but also other reasonable concepts, while current approaches usually fail to explore huge unknown (i.e., combinations verbs and objects). In this paper, 1) we introduce novel challenging task for understanding, which is termed as Concept Discovery; 2) devise self-compositional learning framework SCL) concept discovery. Specifically, maintain an online updated confidence matrix during training: assign pseudo labels all composite instances according the self-training; update using predictions instances. Therefore, proposed method enables on both known concepts. We perform extensive experiments several popular datasets demonstrate effectiveness discovery, object affordance recognition detection. For example, significantly improves performance discovery by over 10% HICO-DET 3% V-COCO, respectively; 9% mAP MS-COCO HICO-DET; 3) rare-first non-rare-first detection relatively 30% 20%, respectively. Code publicly available at https://github.com/zhihou7/HOI-CL .